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FIELD OF INVENTION 

The present invention relates to a method and system for evaluating the quality 
of goods and services based on survey ratings and reports from customers, and more 
specifically for evaluating patient care programs. 

BACKGROUND OF THE INVENTION 

Conventional approaches to access customer satisfaction information generally 
use standard survey forms or questionnaires by mail or telephone. Survey results are then 
tabulated or summarized in order to evaluate the information. However, any project or 
initiative will generally have an overwhelming number of measurement options. 
Pursuing all of them results in data overload and frustration. 
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The compass viewpoint concept is a tool that can be used to help evaluate 
customer satisfaction and focus improvement efforts. This improvement process 
provides a framework for developing a balanced measurement profile for an initiative by 
focusing on a small set of measures. The concept consists of four points which display 
four main outcome measurement categories or areas of measurement. Key measures of 
the initiative are then selected by addressing what variables or subcategories might be 
considered under each of the points. 

In the health care system, the four outcome measurement categories are clinical, 
functioning, satisfaction, and cost. Like a directional compass, the points correspond to 
key aspects of quality and performance for both patients and staff. Clinical includes 
measures of mortality and morbidity, such as signs, symptoms, treatment complications, 
diagnostic test results and laboratory results, and disease-specific measures. Functional 
consists of measures of physical function, mental health, social/role function, and other 
measures of health status, such as pain, vitality, and perceived well-being and health risk 
status. Satisfaction includes measures of satisfaction such as patient and family 
satisfaction with health care delivery processes, a patient' s perceived health benefit from 
care received, and employee satisfaction with work environment. Cost consists of 
measures of direct medical costs (e.g., ambulatory care, inpatient services, and 
medications) and indirect social costs (e.g., days lost from work or normal routine, 
replacement worker costs, and caregiver costs). A detailed description of the compass 
viewpoint can be found in "Clinical Improvement Action Guide," edited by Nelson, 
Batalden, and Ryer, 1998, which is incorporated herein by reference. 

However, this improvement process has several limitations. In depth application 
of the compass viewpoint and evaluation of the information collected need several 



1 measurement methods. These methods include control charts indicating variations in 

2 trends over time, patient preferences, rating scales, benchmarking, cost analysis, and 

3 cross related measurements. In addition, evaluators of a survey assessment may be 

4 interested in detailed reasons behind an answer in a particular situation, such as when a 

5 participant is very dissatisfied with a service or product, while the same evaluator may 

6 not care why a survey participant was satisfied with a different service or product. 

7 Processing stated reasons and such measurement methods using standard techniques can 

8 be difficult, time-consuming, and costly. Thus, there is a need for a computer- 

9 implemented system and method based on the improvement process that allows for 

10 presenting collected survey information in a compass viewpoint framework. 

11 It would also be advantageous to automate the analysis and presentation of the 

12 collected survey information to enable a user to more easily and efficiently review the 

13 collected information. Modern computer and networking technology provide potential 

14 solutions to these problems. Advances in database design, computer processing, and 

15 computer networking all provide means to improve the process of analyzing and 

1 6 evaluating survey information. Accordingly, an adaptable survey presentation procedure 

17 would be useful, one that indicates the depth of measurements on certain topics 

18 depending on the answers given to questions on that topic, and that utilizes computer 

1 9 technology to process calculations and verbal replies. 
20 

2 1 BRIEF SUMMARY OF THE INVENTION 

22 According to the present invention, the data presentation system and method 

23 includes a storage device, a display device, a server, and a network connecting the server 

24 to the display device. The server is programmed to maintain in the storage device a 
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1 database with data information from a data collection, perform calculations on the data 

2 information from the data collection, and produce on the display device graphical data 

3 displays indicating measurement results based on selected population characteristics. 

4 The method and system manage information on a technology platform that fully 

5 integrates data collection results, data computation, and data presentation into one 

6 system. Unlike traditional evaluation processes, the data collection information is also 

7 updated regularly and may be accessed via the internet or other network system, 
i . 8 

9 BRIEF DESCRIPTION OF THE DRAWINGS 

H 10 FIG. 1 is a block diagram showing an improvement process which includes an 

^ 11 embodiment of the present invention; 

r , 12 FIG. 2 is a flow chart showing an embodiment of the data presentation system; 

I _ s 13 FIG. 3 is a diagram showing an exemplary configuration for the system shown in 

6 14 FIG. 2; 

1 5 FIG. 4 is a flow diagram for a method of accessing graphical data displays with 

16 the system shown in FIG. 2; 

17 FIG. 5 is an exemplary embodiment of a user login interface; 

1 8 FIG. 6 is an exemplary embodiment of a user password interface; 

19 FIG. 7 is an exemplary embodiment of a user main interface; 

20 FIG. 8 is an exemplary embodiment of an outcome measurement display with 

21 comparative practice information; 

22 FIG. 9 is an exemplary embodiment of a user management topic interface; 

23 FIG. 10 is an exemplary embodiment of an overall satisfaction display from a 

24 specific management question shown in FIG 9. 
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1 FIG. 1 1 is an exemplary embodiment of the display of FIG. 10 over a specific 

2 time period; 

3 FIG. 12 is an exemplary embodiment of a user subsample interface; 

4 FIGS .13a and 1 3b are an exemplary embodiment of a user performance measure 

5 interface; 

6 FIG. 14 is an exemplary embodiment of a rating display from a performance 

7 measure shown in FIG. 13b; 

8 FIG. 15 is an exemplary embodiment of a user drill-down interface; 

1 1 9 FIG. 16 is an exemplary embodiment of a drill-down display from a drill-down 

Jv 10 question shown in FIG. 15; 

y, 1 1 FIG. 1 7 is an exemplary embodiment of a user boolean search interface; 

s 1 2 FIG. 1 8 is an exemplary embodiment of a boolean search display from a specific 

v 1 3 category shown in FIG 1 7. 

H 14 

15 DETAILED DESCRIPTION OF THE INVENTION 

16 The data presentation system and method are part of an improvement process that 

1 7 involves compiling and presenting data. As illustrated in FIG. 1 , a typical improvement 

1 8 process based on the compass viewpoint framework 1 0 includes several steps in order to 

19 evaluate and improve a project or initiative. These steps include stating the aim of the 

20 initiative and selecting a target population to evaluate 20, defining the delivery process 

21 of the customer initiative being evaluated 22, selecting outcome measures and 

22 determining the performance measurements or subcategories to be measured 24, 

23 designing a survey questionnaire or data collection plan based on the selected 

24 measurements 26, performing the data collection plan 28, computing and analyzing the 
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data collection results 30, and then distributing the results 32. An operator or user then 
utilizes the data presentation system 34 to analyze and evaluate the results of the data 
collection and to implement changes to improve the customer initiative 36. Finally, the 
data collection plan is repeated in order to monitor the impact of the change 38. 

FIG. 2 is a block diagram of a data presentation system (DPS) 34 in accordance 
with one embodiment of the present invention. DPS includes a server system 40 
connected to a centralized storage device or database 42 and a plurality of user devices 
44. The database 42 contains the survey or business data and may be stored on the server 
40 or can be stored remotely from server 40. The database 42 may be further divided into 
a survey information data section, a historical data section, and a comparative practice 
data section and be interconnected to update and retrieve the information as required. 
The user devices 44 are computer terminals including a web browser, which access the 
server system 40 via a network 46, such as an internet, an intranet or other private 
network. The user devices may also include any device capable of interconnecting to the 
network 46 including a web-based hand-held device or other connectable equipment. 

hi an exemplary embodiment, the server 40 authenticates the user for access to 
the data collection and encrypts and de-encrypts transferred data. The server system 40 
is more fully discussed in co-pending application INTEGRATED COMMUNICATION 
SYSTEM AND METHOD, by Nelson et al., Serial Number 09/871,420, incorporated 
herein by reference. The server system 40 may also be further divided into multiple 
servers and multiple databases as discussed in co-pending application PHYSICIAN 
OFFICE VIEWPOINT SURVEY SYSTEM AND METHOD, by McEachen, 
incorporated herein by reference. Co-pending application SURVEY PROCESSING 
SYSTEM AND METHOD, by Rogers, is also incorporated herein by reference. 



FIG. 3 is an exemplary embodiment of a diagram 100 for a computer network- 
based method for utilizing DPS. Processes of the data presentation system involve data 
reduction and the scoring of data variables, the creations of scales, the statistical analysis 
of data, and the creation of data displays to generate the presentation screens viewed by 
the user. The DPS retrieves data information 52 stored in the database, performs 
calculations 54 on the data information, and produces reports 56 showing data 
measurements and data information, which can be viewed on the user's computer via 
graphical display methods. Data information utilized by the DPS include historical 
information 58, including information from medical, financial and billing records, survey 
participation and population characteristic 60, survey questions 62, responses to survey 
questions 64, comparative practice information 66, and other data information as 
specified for a particular survey, data collection plan, user, or improvement initiative. 
Calculations and formulas used to produce the reports include those for producing rating 
scales 68 and statistical process control charts 70, producing histograms and pie charts 
72, multivariate methods for adjusting results for case-mix differences, and multiple 
regression analysis for determining which variables contribute most to explaining 
variations in outcomes (e.g., patient satisfaction). The data reports indicate 
measurements on a selected measurement option including performance measures 74, 
information on survey questions 76, comparative practice benchmarks 78, boolean search 
results 80, and verbatims 82, which may be actual responses to open-ended questions 
asked of survey participants, and other qualitative and quantitative measurements. 

FIG. 4 is a flow diagram 100 that illustrates a method for how a user can 
dynamically navigate the data collection stored in the database in order to directly select, 
analyze and receive graphical data displays. A user logs into the DPS system 102 and 



1 selects a survey 104. The user then selects a measurement option or category to view. 

2 hi the exemplary embodiment, the user may select a measurement option of a specific 

3 outcome measurement category 1 06, a management topic 1 08, or a performance measure 

4 110. Depending on the specific user or survey, each of these selections may have further 

5 subcategories and various reports to choose from. After a selection is made, the DPS 

6 retrieves the data information, performs any necessary calculations, and produces the 

7 desired report 112. The report is then displayed 114 on the user's computer using graphic 
y, 8 user interface (GUI) features or other graphical data display methods. The user may 
Q 9 continue to view and evaluate data by selecting other categories or may first stratify data 
g 1 0 1 1 6 by a number of subsample populations 118 before selecting another category. The 
\I 1 1 graphical data display may also be printed out at any time by the user's printer (not 
y, 1 2 shown). The user may also select another survey at anytime while logged into the DPS . 
Q 13 FIG. 5 is an exemplary embodiment of a DPS user login interface 200. The user 

14 login interface 100 facilitates access to the DPS by prompting the user to log into the 

15 DPS. The user is prompted to enter an organization 202, a username 204, and a valid 

1 6 password 206 to gain access to the DPS. Upon accessing the DPS for the first time, the 

1 7 user is also prompted to change the initial password to a customized password as shown 

18 in FIG. 6. In the user password interface 210, the user is prompted to enter the current 

1 9 password 2 1 2, the user' s birth year 2 1 4, a new password 216, and the new password a 

20 second time 218 in order to confirm the new password. The user is also prompted to 

21 enter a challenge question 220, an answer to the challenge question 222, the answer to 

22 the challenge question a second time 224 in order to confirm the answer, and the user's 

23 email address 226. In the exemplary embodiment, if the birth year 214 is answered 
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1 incorrectly, the user will not be able to change the password. The challenge question 220 

2 will be asked if the user requests any information about the user's existing password. 

3 FIG. 7 is an exemplary embodiment of a DPS user main interface 230. The DPS 

4 user main interface 230 lists an outcome measurement menu 232 and a main selection 

5 menu 234 from which the user may select categories to view and evaluate reports. In the 

6 exemplary embodiment, the outcome measurement menu 232 includes four main 

7 outcome measurement categories of Clinical 236, Functioning 238, Satisfaction 240, and 
u 8 Cost 242. The main section menu 244 includes a Home category 244, a Key Questions 
O 9 or management topic category 246, a Data View category 248, a Select Subsample 
H 10 category 250, and an Options category 252. Additional main section menu 234 category 
*T 1 1 options may be provided depending on the specific user. Selection of the Home category 

jj\ 12 will bring the user back to the user main interface 230; selection of one of the other 

is* 

Q 13 categories listed above provides more detailed user interfaces. The Options category 252 

ill 

O 14 contains other options and interfaces unrelated to the other main selection menu 

"m 

15 categories. The Options category includes changing a user's password, news and changes 

16 concerning the DPS, and participation information for a survey. 

1 7 FIG. 8 is an exemplary embodiment of an outcome measurement category display 

18 260. The display shows the organization's survey response results in the form of 

19 thermometer readings 262 for the Functioning 238 and Satisfaction 240 outcome 

20 measurements categories against a comparative group's information. The Functioning 

21 category 238 includes the measurement domain or major domain of care of General 

22 Health 264. The Satisfaction category 240 includes the measurement domains of 

23 Initiating Care 266, Receiving Care 268, Billing & Discharge 270, and Overall 

24 Satisfaction 272 . Each of the thermometer readings 262 represent a percent maximum 
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achievable score. This score represent the average score for the respondents where the 
lowest possible score is zero and the highest possible score is one hundred. A legend 274 
is included to assist the user in evaluating the display. The range on each of the 
thermometers indicates the 10 th to 90 th percentile range of scores for the organization. 
The outcome measurement display 260 may show any number of outcome measurement 
categories depending on the user or survey. The display 260 may also show response 
results against other regions within the organization or multiple comparative groups. 

FIG. 9 is an exemplary embodiment of a user management topic interface 280, 
which is displayed after selecting the Key Questions category 246. The management 
topic interface 280 lists key questions 282 that are relevant to administration or 
management in order that the user can quickly identify main trends in the organization. 
In one embodiment, the topics include, but are not limited to, overall satisfaction 284, 
overall satisfaction during a specific time period 286, how overall satisfaction has 
changed over time 288, responses during a specific time period 290, satisfaction 
compared by site during a specific time period 292, participation characteristics 294, and 
viewable survey topics 296. FIG. 10 and FIG. 1 1 illustrate exemplary embodiments of 
displays generated from the topics of overall satisfaction 284 and how overall satisfaction 
has changed over time 288, respectively. In FIG. 10, a bar chart 300 shows the 
respondent results by a scale 302, with zero being the worst to ten being the best, and the 
frequency percentage 304 of the rankings. A control chart 310 that displays the 
organizational data for overall satisfaction over time is shown in FIG. 1 1 . The date of 
any data point 316 can be identified by moving the mouse pointer to the data point in 
question. In the display, the mouse was pointed to the upper most data point, and the date 
of 5/23/2000 appeared. 
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1 FIG. 12 is an exemplary embodiment of a user subsample interface 330, which 

2 allows the user to stratify data by a number of subsample populations or population 

3 characteristics. Subsample populations 332 include data range, age group, gender, site, 

4 diagnostic group, and any other population characteristic that would be used in a specific 

5 survey. The DPS default is set to select all subsample population options 334 and 

6 automatically displays all data unless the user selects a specific subsample population 

7 338. A selected option is noted by a small checkmark in the box next to the option. 

8 Once the user selects one or more population characteristics for an analysis, the user 
p 9 clicks on a finish button (not shown) so that the subsample stratification can be 
H 10 implemented. The subsample options can be changed or viewed at any time by selecting 
" 1 1 the Select Subsample category 250. FIGS. 1 3a and 1 3b are an exemplary embodiment 
s . 12 ofauserperformancemeasureinterface350aand350b, which is displayed after selecting 

|=55!; 1 .... 

13 the Data View category 248. The user performance measure interface allows the user to 

Q 14 view all the performance measure levels of the survey or data collection information. 

15 The hierarchy of the performance measures includes outcome measurement categories 

16 352, measurement domains or domains of care 354, and specific measures 356. Within 

1 7 each outcome measurement category 3 52 are measurement domains 354, and within each 

1 8 measurement domain is a series of specific measures 356 pertaining to the corresponding 

1 9 domain of care. The measurement domains and specific measures may vary for different 

20 surveys or users. Any one of the measures within each domain can be selected and 

21 viewed. FIG. 14 illustrates an exemplary embodiment of the specific measure Doctors 

22 display 370. The text of the actual survey question is displayed in a pop-up box 376 

23 when the user's mouse is moved to the lower part of the graph. When the pop-up text 

24 box is not in use, a sample value for the measure is displayed (not shown). 

11 



1 The DPS provides drill-down question inquiries to key questions when a survey 

2 respondent scores a question as Fair or Poor. This drill-down feature allows the user the 

3 opportunity to more fully understand the specific areas that are linked to the Fair/Poor 

4 scores. The drill-down questioning model is more fully discussed in co-pending 

5 application INTERACTIVE SURVEY AND DATA MANAGEMENT METHOD AND 

6 APPARATUS, Nelson et al., Serial Number 09/871,278, incorporated herein by 

7 reference. FIG. 5 is an exemplary embodiment of a user drill-down interface 400 for the 

8 specific measure Doctors that was accessed by selecting Drill-Down Questions 378 under 

9 Data View category 246. The four drill-down questions related to Doctors care provides 

10 additional insight into the Fair or Poor ratings of the doctors overall score shown in 

1 1 specific measure Doctors display 370. FIG. 16 illustrates an exemplary embodiment of 

12 a drill-down display 420 related to the drill-down question Doctors Explanation 406 from 

13 drill-down interface 400. 

14 The DPS also provides the ability to stratify a performance measure so that 

15 specific issues can be identified. FIG. 17 is an exemplary embodiment of a boolean 

16 search interface 430 for the specific measure Doctors that was accessed by selecting 

17 Boolean Search 380 under Data View category 246. FIG. 18 illustrates an exemplary 

1 8 embodiment of a boolean search display 43 0 which stratified the specific measure Doctor 

19 by a specific grouping Age Group. 

20 The DPS enables users to interact directly with their data results and to select, 

21 analyze, and receive graphical data displays on a subpopulation of interest; the time 

22 period in which this subpopulation received services; the distribution of the results (i.e., 

23 on ratings and reports) with respect to overall satisfaction and its related dimensions and 

24 subdimensions; the variation in the aforementioned results by selected characteristics of 
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1 the customer, the setting, or the service provider; and statistical analysis of trends over 

2 time in results for selected subpopulations . The user also has the ability to determine the 

3 specific factors that are most strongly related to generating overall service satisfaction; 

4 the nature of the qualitative, verbatim comments that customers relate to the level of 

5 satisfaction or improvements in service delivery; and the greatest sources of service 

6 strength and the largest opportunities for improvement in service satisfaction that can be 

7 used to assist in the setting of priorities. Thus, as a result of ongoing quality 

8 improvement efforts, the performance of the service provider should improve and 

9 subsequent surveys should result in higher ratings. As improvements are made, the 

1 0 surveys may be changed or redesign to measure this change or be suspended for certain 

1 1 performance measure. 

12 While the invention has been described with reference to specific embodiments, 

1 3 various changes may be made and equivalents may be substituted for elements thereof 

1 4 by those skilled in the art without departing from the scope of the invention, hi addition, 

15 other modifications may be made to adapt a particular situation or method to the 

16 teachings of the invention without departing from the essential scope thereof. The 

17 present invention herein is not to be construed as being limited, except insofar as 

1 8 indicated in the appended claims. 
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